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Abstract. In this paper we will study three hypotheses proposed by L. I. Rozonoer [T] 
in optimal control theory in order to derive conditions for the existence of an optimal 
control under all initial conditions, and the relationships between Pontryagin maximum 
principle and the dynamic programming method. 

>' 
OO 

P ! 1 Introduction 

qq ' Let us introduce the following optimal control problem considered in pQ: 

O ' 



OCP To minimize the Lagrange cost functional 



T 

/ F{x,u,t)dt {1.1) 

a ■ 



to 



subject to the controlled system 



x = f{x,u,t), (1.2) 

with u(t) G U and the initial state condition 

x(t ) = x°, (1.3) 

where to ond T with t < T are prescribed real numbers. 

In OCP, U C R m is the control domain while the set of admissible controls under 
consideration is the set of all Lebesque measurable selection u(t) G U ( see also (2.12) 
or Remark 4.2); x° = (x°, • • • , x^) T G M. n is the initial state, x = (xi, • • • , x n ) T G M. n is 
the state variable, and / = (fx," • , f n ) T is n-dimensional vector- valued function, where 
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and throughout this paper the superscript T denotes the transpose of a vector or matrix. 
Other technical assumptions on / and F will be given in the following sections. 
The control Hamiltonian for OCP is 

n 

J?(x,p,u,t) := ^2pJi(x,u,t) - F(x,u,t), (1.4) 

i=l 

where p = (pi, - ■ ■ ,p n ) T G K n is the costate variable. 

For any given initial data (x°, r) with r G [to, T) and x° G M n , we introduce the control 
Hamiltonian system 

^ dJ4?(x,p,u,t) 

with the two-point boundary value conditions 

x(t) = x°, p(T) = 0. (1.6) 

Remark 1.1. In order to distinguish the function (1.4) and the system (1.5) with the 
Hamiltonian (4.10) and the canonical Hamiltonian system (4.21), which will be considered 
in Section 4 and very related to these analogues, we prefer to calling (1.4) (and (1.5)) 
the control Hamiltonian (and the control Hamiltonian system) instead of the Hamiltonian 
(and the Hamiltonian system). 

Definition 1.1. A control «*(■) : [r, T] i— ► U is said to satisfy the Pontryagin maximum 
condition on the interval [t,T] under the initial data (x , r), provided that the unique 
solution (x*(-),p*(-)) of the control Hamiltonian system (1.5)-(1.6) corresponding to this 
control u*(-) satisfy 

J4?{x*{t),p*{t),u*{t),t)>Jf{x*{t),p*{t),u,t), VueL/, a.e.te[T,T}. (1.7) 

Related to OCP, the Hamilton- Jacobi-Bellman equation (or called the Bellman equa- 
tion in [I]) is 

-v T + supJ>t?(x,-v x ,u,T) = 0, (x,t) e M n x (t ,T), (1.8) 
with the boundary condition 

v(x,T) = 0, (1.9) 

where and throughout this paper, the partial derivative of a given function (p with respect 
to r G [to,T] or x G M n will be denoted by ip r or ip x , respectively. 

In order to adapt for the optimal control theory, L. I. Rozonoer [1] first give the 
following concept of weak solution to the Hamilton- Jacobi-Bellman equation: 



2 



Definition 1.2. A continuous function V : R™ x [tojT 1 ] ^ 1 w said to be an extended 
solution of the Hamilton- J acobi- Bellman equation (1.8)-(1.9) provided that, for any x G 
W 1 and t G [to,T] there exists a p* with —p* G d + V x (x,r) and a u* G U such that 

JF(x,p*,u*,t) >Jff(x,p*,u,r), VueU, (1.10) 

and 

J?(x,p\u*,t) G d + V T (x,r), (1.11) 
along with the boundary condition (1.9). 

The notation d + V x (x, r) and d+V T (x, r) in Definition 1.2 means the superdifferential 
of the function V(-, •) at the point (x,t) with respect to x and r, respectively. We will 
recall the concepts of superdifferential and sub differential in Section 2. 

In order to derive conditions for the existence of an optimal control under all initial 
conditions, and thereby the relationships between Pontryagin maximum principle and the 
dynamic programming method, L. I. Rozonoer pQ proposed three hypotheses on OCP. 

Hypothesis 1. The existence of an extended solution to the Hamilton- J acobi- Bellman 
equation is necessary and sufficient for the existence of an optimal control under all initial 
data (x°, r). 

Hypothesis 2. The extended solution of the Hamilton- J acobi-Bellman equation exists 
if and only if for every initial data (x°,t), there exists a unique control satisfying the 
Pontryagin maximum condition. 

Hypothesis 3. If for every initial data (x°,r), there exists a unique control satisfying 
the Pontryagin maximum condition, then this control is optimal. 

Just as emphasized in [H E] that the concept of solution to the Hamilton- Jacobi- 
Bellmen equation need to be generalized to ensure that certain general hypotheses could 
be given on the condition for the existence of an optimal control under all initial data, 
and as a result the relationships between Pontryagin maximum principle and the dy- 
namic programming method. In this approach, [31 H] generalized the concept of solution 
of the Hamilton- Jacobi-Bellmen equation to help demonstrating the necessary and suffi- 
cient conditions for minimization of a functional not only for nonsmooth cases, but also 
for the case where there is even no optimal control, and provide a possibility for investi- 
gating control design. On the other hand, many works such as [3 [6] and the references 
cited within devoted to the relationships between Pontryagin maximum principle and the 
dynamic programming method directly or in the framework of viscosity solution theory of 
the Hamilton- Jacobi-Bellmen equation. There are rich references related to this approach 
(see [Hi]). 

In this paper, we will only focus on the problem OCP in order to study these above 
hypotheses. 
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The rest of paper is organized as follows: In Section 2, we will study Hypothesis 1. 
First, it will be considered the concept of the extended solution defined by L. I. Rozonoer. 
Under some mild technical assumptions, the Bellman function is just right an extended 
solution to the Hamilton- Jacobi-Bellmen equation. Second, one example will be given to 
show that the Bellman function is not an extended solution but a viscosity solution to the 
Hamilton- Jacobi-Bellmen equation, which indicates the application range of the extended 
solution in some sense. Finally, one counterexample will be given to verify that OCP may 
have no optimal controls under some initial data (x°,t) even that the Bellman function 
is a classical (C 2 smooth) solution to the Hamilton- Jacobi-Bellmen equation, which is 
verified to be an extended solution as well. In Section 3, we will study Hypothesis 2. 
Two counterexamples will be given to show that there are many optimal controls under 
every initial data (x°, r) even that the Bellman function is a classical (C 2 smooth) solution 
to the Hamilton- Jacobi-Bellmen equation, which is verified to be an extended solution as 
well. On the other hand, all optimal controls satisfy the Pontryagin maximum condition. 
In section 4, we will study Hypothesis 3. First, it will be given the necessary and 
sufficient condition for the differentiability of the Hamiltonian. Then, main results will be 
established that Hypothesis 3 holds true under some technical assumptions of regularity 
on the data, through the existing relationships between the Hamiltonian system and the 
Hamilton- Jacobi equation. In Section 5, the conclusions will be given. 

2 On the extended solution and Hypotheses 1 
2.1 On the concept of the extended solution 

First, we recall some related concepts and results in the theory of the Hamilton- Jacobi- 
Bellman equations. 

Let n be a positive integer. We denote by the operation (•, •) and || • || the inner product 
and norm in MJ 1 . 

The following definition is combined from [7], [8] and [5J, etc. 

Definition 2.1. For a continuous function p> : IR n x [t , T\ i— > WL, define that 



d + tp XyT (x,r) and d-.tp XtT (x,r) are called the superdifferential and subdifferential of p> at 
(x,t), respectively; d + ip x (x,r) and d^tp x (x,r) are called the partial superdifferential and 
subdifferential of ip at (x, t) with respect to x, respectively; d + p T (x,r) is called the partial 
superdifferential of ip at (x,r) with respect to r. 




d + cp x>T (x,T) := {(p,q) G M n+1 | lim sup (2/ji) _ (a . r) 
d_p> X:T (x,r) := {(p,q) G R" +1 | liminf^t^^) ■ 



(2.1) 



for a given (x,r) G M" x (t ,T). 
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Remark 2.1. We can define the right superdifferential d + (p XjT+ (x,T), right subdiffer- 
ential d_(p X)T+ (x, r) and partial right superdifferential d + (p T+ (x,r) with respect to r at 
(x,t) G R n x [t Q ,T) by restricting t { r in (2.1). Analogously, define the left superdif- 
ferential d+ip X)T -(x, t), left subdifferential d-ip x>T -(x,r) and partial left superdifferential 
d + Lp T _(x,r) at (x,r) 6l"x {t ,T} by restricting 1 1 r in (2.1). 

From the above definition, it can be easily deduced that 
Lemma 2.1. Lei 95 : R n x [i , T] 1— > R 6e continuous. It holds that 

(a) i/ (p, g) G d + (f XtT (x, r), then p G d + (p x (x, r) and g G d+(p T (x, r); 

(b) i/ (p, g) G 9 + ^a;, T+ (a;, t Q ), then p G d+(p x (x, t Q ) and q G 9+^ T+ (x, t ); 

(c) if (p,q) G d+ip^r-fajT), then p G (A.y? x (x, T) and g G 9 + ^ T _(x,T). 

The definition of the viscosity solution to the first order PDEs is first given by Crandall 
and Lions [9]. We can also refer to [7j, [8], \±U\ and [6], etc., for the following definition. 

Definition 2.2. A continuous function <p : R n x [to,L] 1— > R is called a viscosity sub- 
solution of the Hamilton- J acobi- Bellman equation (1.8) provided that, for any (x,t) G 
R n x (t ,T), 

- q + sup Jf(x, -p,u,r) < 0, V(p, q) G d+<p x>T (x, r); (2.2) 

^4 continuous function Lp : R n x [to ? L] h> R is called a viscosity supersolution of the 
Hamilton- J acobi- Bellman equation (1.8) provided that, for any (x,r) G R n x (t ,T), 

- q + sup J^(x,-p,u,r) > 0, V(p, g) G d_(p x>T (x, r). (2.3) 

uet/ 

Finally, Lp is called a viscosity solution of the Hamilton- J acobi-Bellman equation (1.8) 
if it is simultaneously a viscosity sub- and supersolution. In addition, if Lp satisfies the 
boundary condition (1.9), then Lp is called a viscosity solution of the Hamilton- J acobi- 
Bellman equation (1.8)- (1.9). 

Denote R + = [0, +00). For any R > 0, we denote by B R (R n+1 ) (or B R (R n )) the open 
ball in R n+1 (or R n ) with a radius R centered at 0. 

Definition 2.3. Consider a convex subset A C R" +1 (or W 1 ). A function Lp : A 1— > R is 
called semiconcave if there exists a function uo : R + x R + 1— > R + satisfies that 

( u(r,d) <uj{R,D), W<R,d<D, 

\ \im D ^ 0+ u(R,D) = 0, VA>0, ^ ' ' 

such that, for every R > 0, A G [0, 1] and any f , 77 G A" n 5 R (R n+1 ) for A n B fl (R n ) 

a^(o + (1 - \Mv) - m + (i - m < a(i - A)ne - ^ika, - ^id- (2.5) 

VKe caZ/ i/ie a&oi>e function u a modulus of a semiconcavity of Lp. 
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Remark 2.2. Obviously, if <p : R n x [t , T] i— > R is semiconcave, then both t ) : 
R n i — > R and </?(-, T) : R n i— > R are semiconcave. 

By Rademacher's theorem ([H], Ch.5, p. 281), if tp : R n x [£ , T] i— > R is locally Lipschitz 
continuous, then ip is differentiate almost everywhere in R n x [iojT 1 ]- Meanwhile, 

D*<p(x,r) := { lim <p XtT (x i} Ti) \ V{<p) 3 (x^n) -> (x,r) 

2— >+oo 

such that lim (p XT ( x h T i) exists}, 

is nonempty at all (x, r) £ R n x [£ , T], where 

p( V9 ) ;= {(x,t) E R n x (t ,T) | is differentiable at (x,t)}. 

Lemma 2.2. Let p> : R ra x [t , T] i— > R 6e locally Lipschitz continuous. If ip is semiconcave, 
then it holds that 

(a) /or a// (x,r) G R n x (t ,T), 

d + <p X)T (x,r) = coD*cp(x,r) ^ 0, 
where the operation " co " denotes the convex hull; 

(b) 

d+<p XjT+ (x, t ) D coD*cp(x,t ) ^ 0, 

and 

d+p x ,T-(x,T) D coD*cp(x,T) 7^ 0. 

Proof Part (a) is one part of Theorem 3.3.6 in [5] or in [7]. Part (b) follows easily 
from the locally Lipschitz continuity and the semiconcavity of p> on R n x [to, T], similar 
to the proofs of Proposition 3.3.1 and 3.3.4 in [8]. □ 

Remark 2.3. The semiconcavity of p> : R n x [t Q , T] > R implies the locally Lipschitz 
continuity only on R n x (t ,T). (see jglCj) 

In this section, we will need some technical assumptions on / : R n x U x [to, T] i— > R n 
and F : R n x U x [t Q , T] i-> R as follows: 

(HI) Both / and F are continuous, and there exists a constant M > such that 

||/(0,u,*)||<M, V(u,t)eUx[t ,T], (2.6) 

and 

\F(0,u,t)\ < M, V(u,t) eU x[t ,T}. (2.7) 
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(H2) Both /(■, u, ■) and F(-, u, ■) are locally Lipschitz continuous on R™ x [t , T] uniformly 
in u G U, i.e., there exists a nondecreasing function L : R + t— > R + such that, for any 
(x,t), (y,s) G B R (R n ) x [t ,T\, and any ueU, 

\\f(x,u,t)-f(y,u,s)\\<L(R)(\t-s\ + \\x-y\\), (2.8) 

and 

\F(x,u,t) - F{y,u,s)\ < L(R)(\t -s\+ \\x - y\\). (2.9) 

(H3) There exists a modulus u : R + x R + i— > R + satisfying (2.4) such that, for any 
A G [0, 1] and (x, t), (y, a) G B R (R n ) x [t , T], and any u E U, 

\\Xf(x, u, t) + (l- X)f(y, u, s) - f[Xx + (1 - X)y, u, Xt + (1 - X)s] \\ 

< A(l - X)(\t -s\ + \\x - y\\)uj(R, \t-s\ + \\x - y\\). 1 ' ' 

(H4) F(-, u, ■) is semiconcave on ~R n x [t , T], i.e., there exists a modulus u : M + x M + i— > 
M + satisfying (2.4) such that, for any A G [0, 1] and (x,t), (y,s) G 5 K (R") x [t ,T], 
and any u £ U, 

XF(x, u, t) + (1 - X)F(y, u, s) - F[Xx + (1 - A)y, u, At + (1 - A)a] 

< X(l - X)(\t - s\ + \\x - y\\)u(R, |t-s| + { ' ' 

Remark 2.4. (H3) holds true in particular when / is continuously differentiable with 
respect to (x, t) uniformly in u. More precisely, if we assume that there exists a modulus 
u : R + x R + h+ R + satisfying (2.4) such that, for any (x, t), (y, s) G B R (R n ) x [t , T], and 
any u G U, 

\\f t (x,u,t) - f t (y,u,s)\\ + ||/*(a?, u,t) - f x (y,u,s)\\ < u(R,\t-s\ + \\x-y\\). 

Conversely, under the assumption (H2), it follows from Proposition 1.1.13 in [10] that, 
(H3) implies that / is continuously differentiable with respect to (x,t). 

Throughout this paper, we define the set of all admissible controls under any given 
initial time r G [iojT 1 ], as follows: 

U{t) := {«(•) : [t, T] i— > U\ u(-) is Lebesgue measurable}, (2-12) 

and denote by U simply for U(t ). For any given initial data (x°,r) G R™ x [t ,T], 
(•)) is the unique solution of the control system (1.2) under the initial state 
condition x(t) = x° and the control u(-) G W(r), which we will only denote by x(-) for 
short if without confusion. The value function (or called the Bellman function in [1]) 
V(; •) : R n x [t , T] ^ R is defined by 

V(x°,r):= inf [ F(x(t),u(t),t) dt, (2.13) 

where x(-) is the solution of (1.2) under the initial state condition x(r) = x° and the 
control u(-) eU(t). 

Similar to Theorem 4.1 in [T2] (or [TUJ [S] etc.), it follows that 
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Proposition 2.1. Assume that (H1)-(H2) are satisfied. Then the value function V(-,-) 
is locally Lipschitz continuous on R n x [t Q ,T]. 

Similar to Theorem 3.2 in [12] (or [TOj E] etc.), it follows that 

Proposition 2.2. Assume that (H1)-(H4) are satisfied. Then the value function V(-,-) 
is semiconcave on R n x [to,? 1 ]- 

Remark 2.5. The technical assumptions (H1)-(H2) in Proposition 2.1 and (H1)-(H4) 
in Proposition 2.2 can be weaken in some approaches, for example, the consideration of 
the cases with unbounded control variables in [12J, etc. 

Theorem 2.1. Assume that (H1)-(H4) are satisfied. If the control domain U is compact, 
then the value function V(-, •) : R n x [£ , T] i— > R is an extended solution of the Hamilton- 
Jacobi- Bellman equation (1.8)-(1.9). 

Proof It is well known that, V is the unique viscosity solution to the Hamilton- 
Jacobi-Bellman equation (1.8)-(1.9). 

Proposition 2.1 and 2.2 yields that the value function V is locally Lipschitz continuous 
and semiconcave on R n x [to, 7]- 

By Rademacher's theorem ([TT], Ch.5, p. 281), the locally Lipschitz continuity of the 
value function V implies that V is differentiate almost everywhere in R n x (to, T). Hence, 
for any given (x, r) G R n x [to, T], there exists a sequence of {(x{, Tj)} C M. n x (t , T) such 
that 

(xi,Ti) — > (x,t), as i — > +oo, 

and V is differentiable at all (x+, Tj). According to Proposition 1.9 in [7] (or Theorem 1 in 
[TT] . Ch.10, p. 545), the value function V satisfies the Hamilton- Jacobi-Bellman equation 
(1.8) at (xj,Tj) in the classical sense, i.e., 

- V T (Xi,Ti) + SUp Jt?(Xi, -V x (Xi,Ti),U,Ti) = 0. (2.14) 

Meanwhile, due to the compactness of U, there exists a sequence of {ui} C U such that 

- V T (xi,Ti) + J4?{xi, -V^x^n),^,^ = 0, (2.15) 

and there exists a subsequence of {ui} (still denoted by themselves without loss of gener- 
ality) such that 

Ui — > u* G U, as i — > +oo. 

Denote that 

Pi ■= -V x (xi,Ti), q { := V T (xi,n). 

It follows from Lemma 2.2 that, there exists a subsequence of {(— Pi,qi)} (still denoted 
by themselves without loss of generality) such that 

( d + V XjT (x,r), if rG (t , T), 
V XtT (xi,Ti) = (-p.-, ft) -> (~p*,q*) G < d + V XtT+ {x,t ), if t = t Q , (2.16) 

[ d + V x<T .(x,T), if r = T, 

as i — * +oo, where Mr, r (^i,Ti) is the derivative of V(-, ■) with respect to (x,r) at (xj,Tj). 
By Lemma 2.1, combining (2.15) and (2.16) yields the conclusions. □ 
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2.2 An example for the concept of the extended solution 



Example. Let the control domain be U — [0, 1]. For any given initial data (x , r) with 
r G [0, 1) and x° G K. Consider the following linear one-dimensional control system 

^ = s(t)+«(t), fe[r,l], (2.17) 

with the initial condition 

x(t) = i°GR, (2.18) 
and let the associated cost functional be 

J(x°,t;u(-)) = J \x(t)\dt, (2.19) 

where the set of all admissible controls is 

U(t) := {«(•) : [r, 1] h- > [/| «(•) is Lebesgue measurable}. (2.20) 
Obviously, this optimal control problem has a unique optimal control 



I 0, if x° > 0, 

" ^ i '. ir.r". o.d • ,■»),■ < i. (2 - 21) 



while 



U[t) -^ 0, t G [ r -ln(l + x°),l], 



if x° < and (1 + x Q )e l ' T > 1. 

The corresponding Hamilton- Jacob i-Bellman equation is 

-f + sup ueC7 {(x + «)g - \x\} = 0, (x, r) G R x (0, 1), 
v(x,l) = 0. 



'2.23) 



Obviously, the Hamilton- Jacobi-Bellman equation (2.23) has a unique viscosity solu- 
tion 

( x(e 1 - T - 1), if x > 0, 

V(x,r) = < 2 + x-r- (l + ^e 1 ^, if x < 0, (1 + x)e 1 ~ T < 1, (2.24) 
[ x - ln(l + x), if x < 0, (1 + x)e 1 ~ T > 1, 

which is just the value function. 

For any given r G (0, 1), we have 

f ^(e 1_T0 - 1), if x > 0, 

which implies that 9+V^(0,ro) = and 9+T4 )T (0, to) = 0. 

Hence V(-, •) defined by (2.24) is a viscosity solution but not an extended solution of 
the Hamilton- Jacobi-Bellman equation (2.23). 

We notice that V(-, •) is not a semiconcave function according to Lemma 2.2. 
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2.3 An example for Hypotheses 1 

We consider the following example of optimal control problem, which is adapted from [13] 
(Ch.3, p.246). 

Example. Let the control domain be U = [—1,1], and the set of all control variables be 

U := {«(■) : [0, 1] i — ► U\ u(-) is Lebesgue measurable}. (2.26) 

Consider the one- dimensional control system 

dx 

— = u(t), £G[0,1], (2.27) 
with the initial state condition 

x(0) = i° G M, (2.28) 
and let the associated Lagrange type cost functional be 

J(x ; «(•))= / [x 2 (t) - u 2 (t)} dt. (2.29) 



For this optimal control problem, the corresponding Hamilton- Jacobi-Bellman equa- 
tion is 



-f + lil-* 2 + 1 = °> (v)e«x(o,i; 

v(x,l) = 0. 



^2.30) 



It is easy to verify that, the Hamilton- Jacobi-Bellman equation (2.30) admits a C 2 
solution 



V(x,t) = < 



l[x 3 - (x + t- l) 3 ] +t- 1, ifx + r>l, 
7X 3 + r-l, ifx + r<l,x>0 



:! - (2.31) 

; - ' -i— t — i ii — /• <- i / - <• i i 

3" 



ix 3 + r - 1, if — x + t < 1, £ < 0, 

i r_„3 _ , ^ _ i \3i 



3 [-x 3 - (-X + r - l) 3 ] + r - 1, if-x + r<-l, 



which is just the value function related to this optimal control problem. Certainly, this 
solution is also an extended solution of the HJB equation since the control Hamiltonian 
is 

J4?(x,p, u, t) = pu — x 2 + u 2 , (2.32) 

and 

max J4?(x,p, u, t) = \p\ — x 2 + 1, (2.33) 
ue[-i,i] 

is attainable at u — — 1 or m = 1. 

However, it can be proved similarly to [13J (Ch. 3, p. 247) that there exists no optimal 
control under the initial data (0,r) with r G [0, 1). In these cases, V(0,r) = — 1 + r, 
which is not attainable. In fact, there exists no optimal control under any initial data 
(x°, r) with r G [0, 1) and \x°\ + r < 1. 
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3 On Hypotheses 2 



3.1 The first example 

Consider the special cases of OCP with the integrand in the cost functional (1.1) satisfies 
that 

F(x,u,t) = C, V(x,u,t) EW 1 xU x[t ,T], (3.1) 

for some constant CgR. Meanwhile, let / : W 1 x U x [to, T] i— > MJ 1 satisfies the following 
assumption: Both / and f x are continuous on ¥L n x U x [to, T], and there exists a L > 
such that 

\\f(0,u,t)\\^L el»x[/x [t ,T). (3.2) 

\\f x (x,u,t)\\ < L, 

In these cases, any control u(-) : [r, T] i— > J7 is an optimal control under the initial 
data (x°,t), while the Hamiliton-Jacobi-Bellman equation 

n 

-v T + sup[- it, r)^ - C] = 0, (3.3) 

ueu ^ 

with the boundary condition 

v(x,T) = 0, (3.4) 

admits a classical solution V(a:, r) = C(T — r), which is obviously an extended solution 
of (3.3)-(3.4). 

On the other hand, according to Pontryagin maximum principle ([E]), any optimal 
control u(-) satisfies the Pontryagin maximum condition. 

Therefore, the Hamiliton-Jacobi-Bellman equation (3.3)-(3.4) has an extended solution 
while more than one controls satisfies the Pontryagin maximum condition. 

3.2 The second example 

Consider the special case of OCP with U = M m and the quadratic cost functional, which 
is defined by 

F(x,u,t) = u 1 Su, (3.5) 

where S G W mxm is nonnegative semi-definite, i.e., u 7 Su > for any u e W 71 and there 
exists u ^ such that UqSu = 0. Meanwhile, let / : R n x U x [to, T] i-> R n satisfies the 
same assumption in the previous example. 

In this case, the Hamiliton-Jacobi-Bellman equation 

- o- + ^vi-yZMxyUyT)- u T Su] = 0, (3.6) 

CT U £U • OXi 

1=1 
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with the boundary condition 

v(x,T) = 0, (3.7) 

admits a classical solution V(x, r) = 0, which is obviously an extended solution of (3.6)- 
(3.7). 

For any constant feel, the control u(t) = kuo is an optimal control under any initial 
data (x , t). According to Pontryagin maximum principle ([H]), any optimal control u(-) 
satisfies the Pontryagin maximum condition. 

In general, Hypothesis 2 may not hold for OCP. 

4 On Hypotheses 3 

4.1 Some Preparations 

Let (j) : R™ x U i— > R be a given function. 

$(x) := sup{0(x,?i)}, Vi 6 M", (4.1) 

is an extended function, i.e., taking values in R U {+00}. For any x G R n , 

Mix) = argmin<^(x, -u) := {u G £/| <fi(x,u) = $(x)}, (4.2) 

net/ 

which is possibly empty. 

Lemma 4.1. Assume that $(x) zs finite for all x G R n , and there exists a modulus 
uj : R + x R + i — > R + satisfying (2.4) stzc/i i/iai ; 

|0(x,w)-^(y,u)|<w(i2,||x-y||), Vx,y e B R (R n ), u e U, (4.3) 

where B R (M. n ) denotes the open ball in R n with a radius R > centered at 0. 

cL$(x) D d-6 x (x, u), „ „ _,. . 

_ , V - ^ ' /or a// u G M(x). 4.4 

d +( f) x (x,u) D d+$(x), 

Proof Since — $(x) = mf ue u[—(f)(x,u)], applying Lemma 2.11 in [7j yields the con- 
clusions. □ 

Lemma 4.2. Assume that U is compact, and <p '■ R n x U 1— > R satisfies (4-3), and 

(A) <fi(-,u) is differentiable at x G R uniformly in u EU , i.e., there exists some modulus 
u)\ : R + 1 — > R + with 



such that 



gmw 1 (i2) = 0, (4.5) 

-Tt — >U 



(x + Ax, — 0(x, u) — — — (x, it) Ax I < II Ax 11^! (II Axil), (4.6) 

ox 



/or smaZ/ Ax ond a// u G [/; 
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(B) -^-(f)(x, ■) '. U i — > R n zs continuous; 

(C) •) : U i— > R is lower semicontinuous. 
Then M(x) ^ 0, and 

a_$(x) = coy(x), (4.7) 

where Y(x) := w G M(x)}/ 

_ _ , s f ifY(x) is a singleton, , , . 

<9 + $(x) = ^ w x . (4.8) 

[ (/), zj r (x) is not a singleton. 

In particular, $ is differentiable at x if and only if Y(x) is a singleton. 

Moreover, $ has the directional derivative in any direction v G R n , given by 

d& . . dep.. . . 

— — fx) = max — — (x,Mm = max (p.v). (4.9) 

CW u&M(x) OX ped-$(x) 

Proof Since — $(x) = inf ue j/[— u)], applying Proposition 2.13 in [7] yields the 
conclusions. □ 

Definition 4.1. The function H : R" xl" x [t ,T] ^ R de/ined by 

n 

H(x,p, t) := sup Jf?(x,p, u, t) = sup / \Pifijx, u, t) — F(x, u, t)}, (4-10) 
usu ueu . , 

is called the Hamiltonian related to OCP, where the functions f = • • • , f n ) T '■ R n x 
U x [t , T] ^ R" and F : R" x U x [t , T]\—>R are the data of OCP. 

In this section, we will need some technical assumptions on / : R n x U x [£ , T] \— > R n 
and F : R" x [/ x [t , T] i-> R as follows: 

(H5) Both / and F are continuous, and there exists an absolute modulus u : R + x R + i— > 
R + satisfying (2.4) such that, 

\<j>(x,u)-<f>{y,u)\<u(R,\\x-y\\), Vx, y G B R (R n ), u G C/, (4.11) 

with : R n x i-^ R being 

(p(x,u) = fi(x,u,t), or or f n (x,u,t), or F(x,u,t), (4.12) 

for any given £ G [£o, F]. 

(H6) ^/ : R n x [/ x [t ,T] ^ R nxn and J^F : R n x U x [t ,T] ^ R n are continuous, 
and there exists an absolute modulus uj\ : R + h- > R + satisfying (4.5) such that 
(f) : R n x U ^ R in (H5) satisfy (4.6), for any given t G [t ,T\. 

Proposition 4.1. Assume that (H5) holds, and both f and F are differentiable with 
respect to x G R n . Then it holds that 
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(I) H(x,-,t) is convex; 

(II) // 

A(x,p,t) : = {u G U\Jf(x,p,u,t) = H(x,p,t)} ^ 0, (4.13) 

and H(x, -,t) is differentiable at p G R, then 

f(x,u,t)=c, on A(x,p, t); (4-14) 

If A(x,p,t) 7^ and H(-,p,t) is differentiable atpER, then 
d d 

—f(x,u,t)p-—F(x,u,t) = c, on A(x,p,t). (4.15) 

Proof (I) Let ^f(x,p, u, t) = (f(x, u, t),p) — f°(x, u, t), we have 
H(x, \pi + (1 - \)p 2 , t) = sup Jif(x, Xpi + (1 - \)p 2 ,u,t) 

= sup[A^(x,pi, u, t) + (1 — X)J^(x,p 2 , u, t)] 
< \H(x,pi,t) + (l-\)H(x,p2,t), 

for any A G [0, 1]. 

(II) Lemma 4.1 yields the conclusions. □ 

Corollary 4.1. Assume that f and F satisfy the assumptions in Proposition 4-1- If 
H(-, - ,t) is differentiable and A(x,p,t) ^ at some (x,p) G R™ x R n . Then 

^H(x,p,t) = {f(x,u,t)\ J?(x,p,u,t) = H(x,p,t)} 
-^H(x,p,t) = {£f(x,u,t)p- -^F(x,u,t)\ Jt?(x,p,u,t) = H(x,p,t)}. 

This corollary is just Proposition 3.2 in [T5] . 



Proposition 4.2. Assume that (H5)-(H6) holds, and the control domain U is compact. 
If 

A(x,p,t) := {u G U\J4?(x,p,u,t) = H(x,p,t)} ^ 0, (4.17) 

and 

f(x,u,t)=c, on A(x,p,t), (4-18) 

then H(x, •, t) is differentiable at p G R; If A(x,p, t) ^ and 
d d 

—f(x,u,t)p-—F(x,u,t)=c, on A(x,p,t), (4.19) 
ox ox 

then H(-,p,t) is differentiable at x G R. 

Proof Lemma 4.2 yields the conclusions. □ 
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4.2 Main Results 

It is well-known that, under some convex assumptions of the data / and F, the necessary 
condition - Pontryagin maximum principle is also sufficient (see [6], [TO] etc.). According 
to Theorem 2.5 in [6], we have 

Proposition 4.3. Assume that the control domain U is convex and Jtff(-,p,-,t) : M. n x 
U \— ► R is concave for all (p, t) G M n x [to, T] . Then u* : [r, T] i— > U is an optimal control 
o/OCP under the initial condition x{r) = x° with (x°,r) G M" x [to,T), if and only if 
u* satisfies Pontryagin maximum principle, i.e., 

Jt?(x*(t),p*(t),u*(t),t) = max Jt?{x*(t),p*{t),u,t), a.e. tG[r,T], (4.20) 

where (x* (■) , p* (■)) is the unique solution to the control Hamiltonian system (1.5)-(1.6). 
In particular, Hypotheses 3 is true. 

Remark It is obviously the fact that, Hypotheses 3 is true provided that there exists an 
optimal control under all initial conditions. There exist many references on the existence 
of optimal controls such as the famous Cesari' type conditions [T7], etc.. 

Now we will consider Hypotheses 3 directly through Hamilton- Jacobi Theory based 
on some regularity condition of the data / and F instead of the above convex conditions. 

Definition 4.2. Let : R" x 1" x [t ,T] ^ E be defined in Definition 4.1. The system 

x = o^H(x,p,t), . 
p=-9- x H(x,p,t), P (T) = 0, 1 • J 

is called the Hamiltonian system related to OCP. 

Definition 4.3. The Hamiltonian system (4-21) is called a complete system provided that, 
for any £ G M. n , (4-21) under the terminal condition 

x(T) = e, (4.22) 

has a unique solution (x(-),p(-)) on [to, T}; Moreover, for any sequence of solutions 
(^(•),^(-)) of (4.21) with 

lim (x l (r t ),p l (r t ),T t ) = (£, v ,t) G W n x R n x [t ,T], 

i— >+oo 

it holds that (x l (-),p l (-)) converge uniformly on [to,T] to the solution (x(-),p(-)) of (4-21) 
with (x(r),p(r)) = (£,77). 

Definition 4.4. The Hamiltonian system (4-21) related to OCP is called to have a shock 
at time r G [to,T) if there exist two solution (x l ,p l ) : [to? 7] h- > R™ x R n of (4-21) with 
i = 1,2, such that 



x 1 (r)=x 2 (r), p\r)^p 2 (r). (4.23) 
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According to [TO] (see Ch.5, p.607-610) or it follows that 

Lemma 4.3. Assume that the Hamiltonian system (4-21) is a complete system and also 
has no shock at all. If for every £ G M. n , there exists a control u : [to,T] i— > U such that 
the solution (x(-),p(-)) of (4-21) with the terminal condition 

x(T) = e, (4.24) 

satisfies 

x{t) = f{x{t),u{t),t), a.e te[t ,T], (4.25) 
then V : W 1 x [t , T] ^ R de/med 6y 

V(x(t),t) := [ F(x(s),u{s),s)ds, (4.26) 



is t/te va/we function o/OCP. 

Theorem 4.1. Assume that (H5)-(H6) /io/<is and i/ie Hamiltonian system (4-21) is a 
complete system. If the control domain U is a compact, then Hypotheses 3 holds. 

Proof By Hypotheses 3, there exists a unique control u* : [r, T] i— > U satisfying 
Pontryagin maximum principle under any initial data (x°,t). Thus V : MJ 1 x [t ,T] i— > R 
is well-defined as follows: 

V(x°,r):=[ F(x*(s),u*( y s),s)ds, (4.27) 



where x*(-) is the solution of (1.2) with the initial condition x(r) = x° and the control 
«*(•)• Meanwhile, it follows from (4.16) and the assumptions that, the unique solution 
(x* (■),£>*(■)) of the control Hamiltonian system (1.5)- (1.6) with «(•) = u*(-) is also the 
unique solution of the Hamiltonian system (4.21) with the terminal condition 

x(T) = x*(T). 

The assumptions and Lemma 4.3 tell us that we only need to prove the Hamiltonian 
system (4.21) has no shock at all. Otherwise, if there are two solutions (x*(-),p*(-)) of the 
Hamiltonian system (4.21) with i = 1,2, such that 

x\t)=x 2 (t), p\t)^ P 2 (t), (4.28) 

for some r G [to,T), then it follows from (4.16) and the famous Filippov's Lemma in 
[T8] (known as Measurable Selection Theorem) that there exist two admissible controls 
u % : [t, T] i — > U with i — 1,2, ( i.e., both u l (-) and it 2 (-) are Lebesque measurable ) 
such that both tt 1 (-) and u 2 (-) satisfies the Pontryagin maximum condition together with 
(x 1 (•) , p 1 (•)) and [x 1 (•) , p 2 (•)) , respectively, under the same initial data (x°, r) = (x 1 (r), r). 
This is contradictory to Hypotheses 3. The proof is completed. □ 

Remark 4.2. In this paper, we take the admissible control set at the initial time r G 
[to, T] as U{t) defined in (2.12), i.e., all Lebesque measurable functions on [r, T]. For other 
types of admissible control set such as all piecewise continuous controls, the conclusions 
in this paper are also valid. 
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5 The conclusions 



In this paper, we study in detail three hypotheses on the optimal control theory proposed 
by L. I. Rozonoer pQ. Hypotheses 3 is only considered for the case with the smooth 
Hamiltonian. Now we are considering the case with the non-smooth Hamiltonian. 
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